Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change external docs redirects to new location on the forum #1346

Closed
wants to merge 2 commits into from

Conversation

marmarek
Copy link
Member

@marmarek marmarek commented Nov 10, 2023

Use Discourse's search API to find new location of each page.

Done using this script:

import yaml
import requests
import sys
import time

BASEURL = "https://forum.qubes-os.org"

def main(path):
  with open(path) as f:
      frontmatter = yaml.safe_load_all(f)
      frontmatter = list(frontmatter)[0]
  # rate limit of 20req/min...
  time.sleep(3)
  # https://docs.discourse.org/#tag/Search/operation/search
  r = requests.get(f"{BASEURL}/search.json", params={
          "q": f"{frontmatter['title']} #guides in:title"})

   r.raise_for_status()
  search = r.json()
  if 'topics' not in search or not search['topics']:
      raise Exception(f"No topic found for \"{frontmatter['title']}\"")
  result = search['topics'][0]
  frontmatter["redirect_to"] = f"{BASEURL}/t/{result['slug']}/{result['id']}"
  with open(path, "w") as f:
      f.write("---\n")
      f.write(yaml.dump(frontmatter))
      f.write("---\n")

if __name__ == "__main__":
  try:
      main(sys.argv[1])
  except Exception as e:
      print(f"{sys.argv[1]}: {e!s}")

And called this way:

find external/ -type f -exec update-community-redirect {} \;

Not all pages were found:

external/building-guides/building-archlinux-template.md: No topic found for "Building Arch Linux template"
external/building-guides/building-non-fedora-template.md: No topic found for "Building non-Fedora template"
external/configuration-guides/multimedia.md: No topic found for "How to make a multimedia template"
external/configuration-guides/tips-and-tricks.md: No topic found for "Tips and tricks"
external/customization-guides/fedora-minimal-template-customization.md: No topic found for "Fedora minimal template customization"
external/customization-guides/removing-templatevm-packages.md: No topic found for "Removing template packages"
external/customization-guides/windows-template-customization.md: No topic found for "Windows template customization"
external/os-guides/netbsd.md: No topic found for "How to create a NetBSD qube"
external/os-guides/pentesting/blackarch.md: No topic found for "How to create a BlackArch qube"
external/os-guides/pentesting/kali.md: No topic found for "How to create a Kali Linux qube"
external/os-guides/pentesting/ptf.md: No topic found for "How to create penetration testers framework (PTF) qube"
external/privacy-guides/tails.md: No topic found for "Running Tails in qubes"
external/security-guides/multifactor-authentication.md: No topic found for "Multifactor authentication"
external/troubleshooting/sony-vaio-tinkering.md: No topic found for "Sony Vaio tinkering"

This PR removes those not found. If somebody has a better idea (manually find them?), it can be easily added - the removal is in a separate commit.

This is replacement for #1343

Use Discourse's search API to find new location of each page.

Done using this script:
```python

import yaml
import requests
import sys
import time

BASEURL = "https://forum.qubes-os.org"

def main(path):
    with open(path) as f:
        frontmatter = yaml.safe_load_all(f)
        frontmatter = list(frontmatter)[0]
    # rate limit of 20req/min...
    time.sleep(3)
    # https://docs.discourse.org/#tag/Search/operation/search
    r = requests.get(f"{BASEURL}/search.json", params={
            "q": f"{frontmatter['title']} #guides in:title"})

    r.raise_for_status()
    search = r.json()
    if 'topics' not in search or not search['topics']:
        raise Exception(f"No topic found for \"{frontmatter['title']}\"")
    result = search['topics'][0]
    frontmatter["redirect_to"] = f"{BASEURL}/t/{result['slug']}/{result['id']}"
    with open(path, "w") as f:
        f.write("---\n")
        f.write(yaml.dump(frontmatter))
        f.write("---\n")

if __name__ == "__main__":
    try:
        main(sys.argv[1])
    except Exception as e:
        print(f"{sys.argv[1]}: {e!s}")
```

And called this way:

    find external/ -type f -exec update-community-redirect {} \;

Not all pages were found:

    external/building-guides/building-archlinux-template.md: No topic found for "Building Arch Linux template"
    external/building-guides/building-non-fedora-template.md: No topic found for "Building non-Fedora template"
    external/configuration-guides/multimedia.md: No topic found for "How to make a multimedia template"
    external/configuration-guides/tips-and-tricks.md: No topic found for "Tips and tricks"
    external/customization-guides/fedora-minimal-template-customization.md: No topic found for "Fedora minimal template customization"
    external/customization-guides/removing-templatevm-packages.md: No topic found for "Removing template packages"
    external/customization-guides/windows-template-customization.md: No topic found for "Windows template customization"
    external/os-guides/netbsd.md: No topic found for "How to create a NetBSD qube"
    external/os-guides/pentesting/blackarch.md: No topic found for "How to create a BlackArch qube"
    external/os-guides/pentesting/kali.md: No topic found for "How to create a Kali Linux qube"
    external/os-guides/pentesting/ptf.md: No topic found for "How to create penetration testers framework (PTF) qube"
    external/privacy-guides/tails.md: No topic found for "Running Tails in qubes"
    external/security-guides/multifactor-authentication.md: No topic found for "Multifactor authentication"
    external/troubleshooting/sony-vaio-tinkering.md: No topic found for "Sony Vaio tinkering"
@marmarek
Copy link
Member Author

PipelineRetry

3 similar comments
@marmarek
Copy link
Member Author

PipelineRetry

@marmarek
Copy link
Member Author

PipelineRetry

@marmarek
Copy link
Member Author

PipelineRetry

@andrewdavidwong
Copy link
Member

Unfortunately, it looks like there is a duplication of effort between this PR and #1334. It is not clear which one should be merged or whether they can (or should) both be merged.

@andrewdavidwong
Copy link
Member

@marmarek, any chance you might want to do something like this for the doc index to replace QubesOS/qubesos.github.io#236?

@andrewdavidwong
Copy link
Member

#1334 has been merged instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants